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AMBITIOUS LEADERSHIP: 
A CASE STUDY OF DATA-DRIVEN 
INSTRUCTION AT KIPP COMIENZA 


JILL GROSSMAN AND BRAD CAWN 


USING DATA TO INFORM INSTRUCTIONAL IMPROVEMENT 

At KIPP Comienza Community Prep — an elementary school in Huntington 

Park, California, where 94 percent of students qualify for free lunch, and more 

than 60 percent are English language learners — most kindergarteners start out 

well below pre-reading levels. In 2015, however, just a few years after these students 
walked in the school doors unable to identify the letters of the alphabet, 81 percent 
met or exceeded standards in English on the Common Core-aligned Smarter Balanced 
Assessment Consortium (SBAC) 

assessment; 82 percent did 

so in math. The school ranked Practices Supporting Robust Data-Driven Instruction at 
eighth in Los Angeles in overall KIPP Comienza 

performance (the average Los 
Angeles school had 33 percent 

of students proficient in English, 
and 25 percent proficient in math) 
and first among schools serving 


* Setting goals: The school leaders and teachers use the 
previous year’s assessment data and network goals to set 
targets for student achievement. 


Frequently collecting a variety of data for monitoring 


low-income English language and attending to progress toward goals: Teachers give 
learners. Even more striking, students standards-aligned assessments multiple times 
KIPP Comienza was the highest a week (e.g., end-of-lesson exit slips), every other week, 
performing school in the nation on at the end of each unit, and quarterly. Other data points 
the math portion of the SBAC. include frequent classroom observations and note-taking 

(teachers of students, school leaders of teachers), adaptive 
Principal Shirley Appleman technology, and surveys. 


attributes her school’s success 


anh Applying that data to all school practices for the sake of 
in shifting to college- and career- improving student learning: Data is used to differentiate 
ready standards to the robust instruction (e.g., with across-grade reading groups), and to 


use of data. “I need to make design professional development. 
sure the data show that we are 


making decisions that are best 
for our kids in this moment,” she 
said. Using data at this level of 
consistency and rigor — so that 
it informs instruction every day, 
in every classroom — requires 
substantial planning and orga- 


Creating and regularly revising tools to facilitate frequent 
and consistent data tracking and analysis: Protocols guide 
teacher discussions around data. Online data trackers 
(stored as Google documents) give all staff access to data 
and enable them to analyze data across classrooms, grades, 
and the entire school. 


Creating systems of support and accountability: The 


nization. To ensure that student principal designed the school schedule to allow for frequent 
data drives every decision about collaboration around data among teachers and instructional 
instruction and curriculum at leaders (the principal, deans, instructional coach, and 

KIPP Comienza, Appleman grade-level leads). Biweekly data meetings are held 

and her team maintain several between principals and deans; deans and teachers (both 
essential practices as listed in the one-on-one and in grade-level teams); and among same- 
inset on the right. grade teachers. 
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SETTING GOALS 

Each summer, Appleman and her leadership team set school-wide goals for the following year for the SBAC, 
Measures of Academic Progress (MAP), standards-aligned interim assessments, reading levels (using 
Strategic Teaching and Evaluation of Progress [STEP] or Fountas & Pinnell), special education, and school 
health (attendance and student attrition). To establish these targets, they look to both KIPP LA Schools’ 
goals for the region and KIPP Comienza students’ performance on the previous year’s assessments. The 
higher number guides the target for the upcoming year. For example, KIPP LA recently set the proficiency 
goal for the region at 60 percent proficient on the SBAC English assessment. Since 81 percent of KIPP 
Comienza’s students had scored proficient the previous year, Appleman set a goal of 85 percent proficiency 
for her students. 


Once Appleman and her leadership team set the annual targets, the entire staff works together in the 
weeks before school starts to set benchmark goals for each cycle. “This is where the teachers do the 
calculating and the heavy lifting,” Appleman said. 


The leadership team looks at both school-wide end-of-year targets and student ability at the start of the 
year. Using a protocol as a guide, teachers determine what each student's trajectory would need to look like 
to meet the end-of-year goal. Specifically, they analyze end-of-year targets, assess incoming reading levels, 
define ideal reading growth in each grade level, and examine past achievement. Then, teachers use back- 
ward planning to determine interim benchmarks. As Appleman explained, they ask: “Based on ideal reading 
growth, where should most students be by the end of Cycle 2, and mid-Cycle 4? In an ideal world, what's 
the breakdown you want to see at each benchmark?” For example, at the beginning of a recent school 

year, 22 percent of third-graders were reading below grade level; the third-grade team determined that 13 
students needed to reach a Fountas & Pinnell reading level N (proficient) by the end of Cycle 2 to meet the 
grade target of having 90 percent of students at least proficient in their reading level. 


The teachers are not left to carry out this work alone. Appleman and her deans facilitate the grade-level 
conversations, mindful of the fact that, as Appleman explained, “From a teacher's perspective this can be 
really scary, since they have to be the drivers of this work and of the outcomes.” In the grade-level team 
meetings, she and her deans ask guiding questions and ensure goals are ambitious yet realistic. 


After the first and second cycles, grade-level teams compare students’ actual performance with the targets, 
and begin to balance ambition with reality. This process guides teachers in identifying the interventions 
students need — whether it involves extra guided reading, reassigning a teacher, or morning intervention 
work, Appleman said. “We identify who the focus students are, and we keep track of them during that cycle 
and make sure they are moving up and making progress.” 


FREQUENTLY COLLECTING A VARIETY OF DATA FOR MONITORING AND ATTENDING TO 
PROGRESS TOWARD GOALS 

Appleman and her predecessor created a system for collecting a variety of data to inform instructional dif- 
ferentiation and professional development, and to help identify effective instructional strategies. Teachers 
assess student learning with exit slips multiple times a week, and with every-other-week and quarterly 
end-of-cycle assessments aligned to standards. The deans observe teachers weekly, recording feedback 
and next steps in an online spreadsheet accessible to coaches and the leadership team. Appleman also 
performs her own classroom walk-throughs to identify trends across teachers and subjects. Teachers keep 
running records of some lessons, and Appleman and her team periodically survey staff on school climate 
and the efficacy of professional development. 
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The staff has put significant time and thought into designing each 
assessment, ensuring it is aligned to standards. “It all starts with me 
doing the research and figuring out, ‘What is the SBAC, how is it orga- 
nized?’” Appleman said. “Then | relay that to my staff.” In the 2014-15 
school year, she and her leadership team took advantage for the first 
time of the materials that were available on the SBAC website. They 
examined the blueprint for how the tests are organized, the practice 
materials, and the item specifications. They identified the question 
types and the language that appeared most often or most significantly 
on the SBAC assessments, and correlated those item types to the 
standards. When designing the school’s own assessments and calendar 
of objectives, the staff included those item types and ensured that they 
exposed students to all of the standards over the course of the school 
year. In sum, Appleman and her staff aligned their curriculum map to 
the design of the SBAC exam. 


Teachers create their own assessments using a matrix designed by the 
leadership team that aligns standards with question types. For each 
Common Core standard, the matrix (see Exhibit 1) lists the Bloom's 
Taxonomy level, cognitive function, “stem” — or questions teachers can 
ask to help monitor progress toward mastering standards — and the 
related skills from the standardized assessment. “It is important that 
we plan with the end in mind and have a clear picture of what we want 
students to know and be able to do by the end of the year,” advises 

an assessment criteria document. Assessment design starts in the 
spring for the following year, when the grade-level teams create the 
first two assessments they plan to give in the fall. The sample questions 
included in the matrix facilitate this process, and teachers take the 
tests themselves, thinking through the questions and how they would 
respond. Coaches then review the assessments and give feedback. 


Assessments Used to 
Analyze Student Progress 
Toward Goals at KIPP 
Comienza 


* Daily probes and 
diagnostics (e.g., running 
records) 


* Common formative 
assessments multiple 
times a week (e.g., exit 
slips) 


Every-other-week 
assessments aligned to 
standards 


End-of-unit standards- 
aligned assessments 


Quarterly end-of-cycle 
assessments 


Smarter Balanced 
Assessment Consortium 
(SBAC) assessment 


Guided reading 
assessments every four to 
six weeks with Strategic 
Teaching and Evaluation 
of Progress (STEP) or 
Fountas & Pinnell 


EXHIBIT 1. KIPP COMIENZA STANDARDS - STEM MATRIX (KINDERGARTEN) 


Common Core | Bloom's Cognitive 
Standard Taxonomy Level | Function 


Use frequently | Understanding Evaluative * How can we tell if a word is a noun? 

occurring reasoning * How can we tell if word is a verb/action word? 
nouns and * Which words tell what the people are doing? 
verbs * Can you circle the verb in the sentence? 


Can you circle the noun in the sentence? 

How did you know that _____ was the verb/noun? 
Which verb/noun best completes the 
sentence? How do you know this? 

Write a sentence. Can you circle the verb and 
underline the noun in your sentence? 

How could we correct the noun/verb in this 
broken sentence? 


* 162: Identifies the spoken 
sentence with correct past-tense 
verb usage (text not shown on 
screen). 

* 162: Identifies the spoken 

sentence with correct verb usage 

(text not shown on screen). 

* 166: Identifies a verb to complete 

a given sentence. 

* 167: Identifies the action word in 

a given sentence. 

* 167: Selects the agreement (do). 
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EXHIBIT 2 


Vitals and 
next steps 


CREATING AND REGULARLY REVISING TOOLS TO FACILITATE FREQUENT AND 
CONSISTENT DATA TRACKING AND ANALYSIS 

After they collect assessment and observation data, grade-level teams meet to analyze the data and 
discuss implications for instruction. To offer support and ensure accountability, deans meet with teachers 
one-on-one to discuss what the trends mean for their classrooms and to determine immediate action steps. 


Teachers are expected to go to grade-level team meetings and one-on-one meetings with their deans 
prepared to talk about data. “They should know their students, where they're projected to land on assess- 
ments, and bring in students’ work to show misconceptions,” said Appleman. She expects the deans to be 
just as familiar with the students and their assessment results. During dean coaching sessions with teach- 
ers, which Appleman occasionally observes, Appleman looks for the deans to push teachers to examine 
the data and determine next steps to support students. Appleman meets with her deans at least twice a 
week to talk about strategies, next steps, and relations with and among teachers. Appleman also makes a 
point of becoming very familiar with the data, so she is able to ask coaches during their meetings about the 
performance of particular classes. 


To ensure these data discussions are consistent across the school, KIPP Comienza’s founding principal 
designed a data protocol, which the entire KIPP LA network has since adopted. Appleman continues to rely 


on this protocol, working with her staff to refine it as needed. 


The protocol for team data meetings (see Exhibit 2 for an overview) lays out the following actions: 


Identification and discussion of the contributing factors that led to the assessment results. 
Acknowledgement of growth and identification of individual students and classes of students that are 
struggling with particular skills. Teams look at how students responded to assessment questions and 
discuss whether their struggles seem to be with content or question format. They determine “vitals,” 
or areas in need of improvement. The protocol advises, “Your vitals should signal clear and explicit 
next steps that, without a doubt, increase student achievement.” 

Development of action steps that are measurable and attainable to address areas of struggle. 
Reflection on team dynamics and discussion of areas of support. 


. KIPP COMIENZA DATA PROTOCOL OVERVIEW 


What are the areas of mastery in the 1. Disaggregate the data to determine the trends in student success. 
performance data? What are the areas of a. Identify areas of mastery. 
growth in the performance data? How are we b. Analyze mastery of each area. 
doing against our goals? 2. Disaggregate the data to determine trends in student success roadblocks. 
a. Identify areas of growth. 
b. Analyze missing mastery of each area. 
3. Evaluate distance from academic goals. 
a. Determine overall achievement gaps. 
b. Analyze range of student performance. 


What content and structures are needed to 1. Identify key data points that signal your next steps. 

support greater student achievement? What | 2. Prioritize content for intervention. 

focused action will lead to greater student a. Identify standards or skills most impacting mastery. 

achievement? b. Determine structures and benchmarks that measure progress. 

3. Develop next steps that are measurable, attainable, relevant, and specific. 
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The grade-level teams record and share data in an online spreadsheet, which is updated after each assess- 
ment. Exhibit 3 shows a sample from a first-grade end-of-cycle assessment analysis. 


EXHIBIT 3. KIPP COMIENZA FIRST-GRADE END-OF-CYCLE ASSESSMENT ANALYSIS 


Classroom Performance in ELA by Homeroom* 


P/A#  P/A% A# A% P# P% Bt B% BB# BB % FBB#  FBB% 
A 25 77% 14 47% 9 30% 7 23% 6) 0% O 0% 
B 24 80% 12 40% 12 40% 6 20% 0 0% O 0% 
C 22 76% 9 31% 13 45% 6 21% 1 3% 0 0% 
D DD 76% 15 52% 7 24% 5 17% 2 7% 6) 0% 


* P/A = Proficient/Advanced; A = Advanced; P = Proficient; B = Basic; BB = Below Basic; FBB = Far Below Basic. 


EXHIBIT 4. KIPP COMIENZA FIRST-GRADE CYCLE 2 VITALS 


Cycle 2 English Language Arts Vitals 


Maintain 91 students at P/A (move 21 students to A). 


Move 8 students identified as “bubble” students (those who are 3 points from proficient). 


Cycle 2 Math Vitals 


Move 27 students to Level P/A. 


Keep 60 students at A and P. 


Move 4 students to A. 


In the spreadsheet, teachers list students who did not score proficient, their scores, where the breakdown 
in learning was (the skill, concept, and standard), and ideas for corrective instruction. In a recent analysis 
of a reading comprehension assessment, a teacher listed a few students who scored far below basic: Their 
handwriting was illegible, and they struggled to answer when asked to verbally explain their answers on 
similarities and differences in the text. In a math assessment analysis, a teacher noted that a few students 
did not carefully distinguish between the words “more” and “less,” and other students struggled to transfer 
knowledge about tens and ones place value to a number sentence. The breakdown in understanding, the 
grade-level team noted, was not reading the last page of the test carefully. The teachers planned to address 
these deficiencies with students either one-on-one or in small groups. 


How does all this data work translate to the classroom? For one thing, teachers create reading groups 
across grades based on assessed reading levels (during guided reading time, students move between 
classrooms to get to their groups). Teachers also use data to divide their students into small groups based 
on their level for a particular lesson. At KIPP Comienza, it is common to find one group of students working 
ona lesson at the board with the teacher, and a few other students working independently — reading 

a book, or working through math problems, or at computers using programs targeted to their English 
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language arts or math level. When the teacher is done giving the lesson to one group, students circulate 
so that everyone gets targeted instruction time with the teacher. In a recent kindergarten class, a group of 
students worked with the teacher on identifying words requiring capitalization in a set of sentences and 
writing the sentences correctly, while a smaller group worked with another teacher on capitalization but 
with a focus on learning to read the words in the sentence. The staff make every instructional and student 
support decision using evidence of student progress. 


CREATING SYSTEMS OF SUPPORT AND ACCOUNTABILITY 

Teachers and leaders at KIPP Comienza attribute their data system's success in part to the collaborative 
structures in place at the school. “We all need to talk about the data because we are each responsible for 
each and every student,” Appleman said, a sentiment she expresses often. Appleman prioritizes developing 
relationships and trust among her staff, and has organized the schedule so that teachers not only have time 
to eat lunch together and share prep periods but also have time to participate in several layers of meetings 
to examine and analyze data. The data meetings are “at the core of how we run the instruction,” said one of 
the school’s instructional deans. “These meetings determine the next steps that we take; they tell us what 
our kids need while we are teaching, who the students are who need specific things, and who the teachers 
are who need support.” The frequent meetings, Appleman added, ensure accountability not only within 
grades but also across the school. 


The KIPP Comienza school schedule includes time for the following data meetings: 


* The principal meets biweekly one-on-one with each dean, and once or twice a week with her team, 
which consists of one instructional coach and three deans (for kindergarten, first and second grades, 
and third and fourth grades). 

* Deans meet one-on-one with teachers at least biweekly. 

* Grade-level teams meet biweekly, led by grade-level team leaders. Deans sit in on these meetings, 
participating as needed. 


IN ACTION: USING THE DATA PROTOCOL ACROSS STAFF MEETINGS 

Observation of collaborative meetings among various staff members makes clear how the data protocol is 
used across the school. This section describes a grade-level team meeting, a meeting between a dean and 
a teacher, and a coaching conversation between the dean and the principal, all of which were held over two 
days in February 2016. 
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THIRD-GRADE TEAM MEETING 

The four third-grade teachers held their biweekly meeting to analyze results on the most recent assessment, an 
end-of-cycle test of both math and English language arts. The dean and grade-level team leader had together set the 
objectives for the meeting. 


Using the school’s data step-back protocol, the team started out by discussing the aims for the meeting. They would 
review whether the identified “vitals” — or areas in need of improvement and next steps — from the previous unit's 
step-back had led to higher student achievement in the next unit, and they would draft vitals to address in the next 
unit of study based on this cycle’s assessment results. 


With copies of the math assessment in front of them, they pulled up the math assessment results on their web 
platform (Illuminate). Looking at results for the whole grade, they identified the problem on which students were least 
proficient, and discussed students’ misconceptions. One teacher noted the pattern of responses in terms of which 
students chose which answers. They discussed why students answered in particular ways. They also noted where 
students within and across classes were most proficient. 


The teachers agreed that overall, students had the foundational fraction skills, but some students needed review on 
improper fractions. “When we go into fraction review,” one teacher said, “let's have improper fractions in there, so 
we can talk about why that is not a whole number but is still a fraction.” Another teacher noted that a few students 
confused 1/3 and 3/1. “We need to address that,” she said. 


Another teacher said she thought some students were struggling to take the test on a computer — they could write 
the answers down on paper, but for some reason they were struggling to note the answer online. (This same teacher 
would later discuss this concern with her dean, and together they brainstormed how to support students in using the 
online platform.) 


They similarly analyzed the reading comprehension assessment. They noted that only one class met the goal, and 
asked the teacher of that class what she thought explained her success. “| thank them for showing so much ganas 
[grit], for being on task and showing their work,” she said, noting that she gives them a treat at the end of the day. In 
her class discussions, she further elaborated, she breaks down questions and asks students to explain their answers, 
and she asks other students to explain what they could have added to answer the question successfully. “I’m looking 
at what my students need and adapting the lesson plan that way,” she said. She also added extra time on reading 
comprehension when she saw her students needed it, and encouraged students to come for extra help. 


They looked at the questions that more than 80 percent of students answered correctly, and discussed teaching 
strategies for those. After analyzing her own students’ answers, one teacher identified figurative language as an area 
that needed improvement. They determined next steps for addressing that learning need — such as giving more 
high-level examples — and set deadlines to ensure accountability. 


“The first next step is tackling the skills in guided reading, tackling standards, and then communicate to support,” said 
one teacher. 


Another teacher had already met with some of her lower performing students who did not meet the goal to go over the test. 
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ONE-ON-ONE MEETING BETWEEN AN INSTRUCTIONAL LEADER AND A TEACHER 
The third-grade dean met with one of her team’s teachers to review the results of a recent assessment. “What are 
your overall reflections on the class's performance and the grade-level performance?” the dean asked. 


The teacher said that she was immediately thinking about the outliers, and named two students. “The work looks really 
good on paper, but for my outliers it was about transferring what was on paper to the computer,” she said. Noting that the 
students struggled with multiple-choice questions, she said that doing the work online is a “completely different ballgame.” 


The dean noted that the teacher should create next steps for the students whose performance surprised her, since 
she knew they could do better. 


Said the teacher, “They have good evidence, but they get stuck identifying the right answer.” The teacher told the 
coach she would check in with each student. The dean suggested she needed to identify some strategies to help 
students improve, noting that the data showed students were performing better in other classes. 


“That should be a guiding question for your data meeting tomorrow,” the dean said. 


MEETING BETWEEN INSTRUCTIONAL DEANS AND THE PRINCIPAL 

In mid-February, Appleman and her entire leadership team met to review the results of the regional math assess- 
ments the students had taken the previous week. While teachers and deans were still working to analyze the data, 
Appleman wanted to talk through the preliminary findings and next steps for completing the analysis. 


Typical of a leader practicing strong interpersonal leadership, Appleman started the meeting with celebrations. The 
school’s performance was the best in the region, she said, “which can be attributed to all of our teachers and staff and 
the work you do.” 


Each dean then reported on his or her grade’s successes and focus areas for improvement. The second-grade dean said 
she and her grade-level team would focus on the students who scored at Level 1 and who barely made it to Level 3. She said 
she was also working with the teacher whose class had the lowest assessment scores. Earlier that day, in her one-on-one 
meeting with Appleman, the dean had also noted that she wanted to push the grade-level team to consider how they could 
transfer their success teaching students one standard to teaching the standards students were still struggling with. Noting 
that it would take extra work for some teachers to apply what they did with one standard to the other standards, Appleman 
suggested the dean hold a planning conference with the teachers she knew would need extra support. 


The first-grade dean reported that the previous month's action step — to observe each other in person and through 
videos and provide feedback — had had a real impact. The students’ performance on the assessment had improved: 
They were better able to show their work in multiple ways. She added that she had already talked with the lowest 
performing class's teacher, who told the dean that she was having trouble with higher level questioning and was not 
giving her students enough time to talk in class. “She is my focus in first grade,” the dean said. 


Like her colleagues, the kindergarten dean reported that she was already working with the teacher whose class had 
the fewest high performers and noted that the students in that class had started the school year at a lower level than 
the other kindergarten students. Appleman agreed, pointing out that the MAP data also reflected that. However, 

the dean said, the kindergarteners had overall shown significant progress in deconstructing problems, an area the 
students had struggled with on the previous assessment and that the teachers had focused on over the last couple 

of weeks. “We set them up for success,” she said. The dean pointed out that teachers would have to reteach the 
standard students did most poorly on — sorting — adding that it had been taught early in the year and never revisited. 


Appleman reminded the deans that the next level of analysis would be to see how the assessment overlaps with MAP. 


Appleman called this meeting typical: Everyone knew where their grade levels were, down to each standard, and 
which teachers needed more support. “Their boots are on the ground getting dirty,” she said. 
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QUESTIONS FOR REFLECTION 


In what ways did the practices listed on page one intersect or build upon one another to enable staff to 
use data to analyze and correct their approach to instruction? 


Of the practices described in the case study, which do you think would be most impactful if adopted by 
or expanded at your school? How could these practices be strengthened to have a greater impact on 
teaching and learning at your school? 


What conclusions can you draw from this case study about the kinds of accountability necessary to 
implement school-wide instructional improvement efforts? What ideas does the case study inspire 
about creating systems of support, feedback, and responsibility at your school? 


Vv 


| a NEWLEADERS.ORG © 2016 New Leaders Inc. All rights reserved. 


